Appendix A — Appendix: Data Review and Upload

Published

November 2, 2023

A.1 Introduction

Prior to publishing analysis and interpretation of water quality data, we will ensure that all data that meets QA/QC standards outlined in the current project Quality Assurance Project Plan (QAPP) and is accessible in the appropriate repository.

Water quality data from this project is ultimately destined for the Environmental Protection Agency’s Water Quality Exchange (EPA WQX).

The Quality Assurance Project Plan (QAPP) for this project describes data management details and responsible parties for each step of the data pipeline from observation to repository. The 2021 data preparation and review process is published here as an appendix as an example of the process applied annually to each year’s data.

A.1.1 Year 2021 Water Quality Data

In this appendix we will collate 2021 laboratory data from several sources into a single spreadsheet document with a consistent format. The desired end format is a spreadsheet template provided by the EPA Water Quality Exchange. These template files are available to download from the EPA at https://www.epa.gov/waterdata/water-quality-exchange-web-template-files.

Once the data is collated, it will be evaluated according to a Quality Assurance Checklist (template example provided by the Alaska Department of Environmental Conservation Soldotna office). Field observations that do not meet the quality assurance standards described in the evaluation checklist will be flagged before being uploaded to the EPA WQX.

Data that has been uploaded to the EPA WQX is evaluated biannually by the Alaska Department of Environmental Conservation (ADEC) in their Integrated Water Quality Monitoring and Assessment Report1. The integrated report evaluates available water quality data from the previous five years against Alaska water quality standards and regulations (ADEC 2020).

A.1.1.1 2021 Water Quality Data AQWMS Formatting

The code scripts in this document assemble water quality data from the three analytical laboratories that partnered with Kenai Watershed Forum for this project in 2021:

  • SGS Laboratories (Anchorage, AK)

  • Soldotna Wastewater Treatment Plant (Soldotna, AK)

  • Taurianen Engineering and Testing (Soldotna, AK)



A.1.1.1.1 2021 Metals/Nutrients Lab Results (SGS Labs)

*Note: the chain of custody documents for SGS Laboratories are integrated into the above downloadable PDF files.



A.1.1.1.2 2021 Fecal Coliform Lab Results (Soldotna Wastewater Treatment Plant (SWWTP)/Taurianen Engineering)


A.1.1.1.3 2021 Total Suspended Solids Lab Results (Soldotna Wastewater Treatment Plant (SWWTP))



A.1.2 2021 Provisional Results, Prior to QA/QC Review

Results last updated 2023-11-02

The above data sources have been collated in to a single .csv file (available for download) into a format compatible with the EPA Water Quality Exchange. These data have not yet been evaluated against QA/QC standards following guidance in the current project Quality Assurance Project Plan.



A.1.3 2021 Data QA/QC Evaluation

Prior to upload to the EPA WQX, all water quality data must be checked against a series of standard questions in order to evaluate how quality assurance / quality check (QA/QC) requirements are met. The draft Data Evaluation Checklist Template (available for download below) outlines these questions:

A.1.3.1 Pre-Database


A.1.3.1.1 Overall Project Success

Note: the sequence of questions is structured such that data will be gradually flagged at each step based on the criteria described.

1.) Were the appropriate analytical methods used for all parameters?

Yes. Analytical methods from the approved 2020 QAPP were employed.


2.) Were there any deviations from the sampling plan?

All sites were visited as planned on 5/11/2021 and 7/27/2021. Most intrinsic water quality parameters measured with instruments (pH, dissolved oxygen, conductivity, turbidity) were not measured.


3.) Were field duplicates, blanks, and/or other QC samples collected as planned?


To see a table comparing planned vs actual results for 2021, download and view the csv file linked above.

From the above table we can see that there are deviations between planned results and actual results available. These reasons for the deviations are known and are attributable to two causes:

Cause 1: The Spring 2021 Chain of Custody (COC) from KWF to SGS was completed erroneously. The COC specified for 200.8 analyses (dissolved metals) to be complete for all sites (when they should have stopped upstream of Morgan’s Landing RM31), and it also specified for 200.7 analyses to stop upstream of Morgan’s Landing (when they should have been performed for all sites in the project).

As a result, for Spring 2021 total metals data will be unavailable for sites upstream of the Morgan’s Landing RM31 site.

Cause 2: For Summer 2021, the SGS laboratory performed the 200.8 analyses (dissolved metals) for all 27 analytes available for the method; instead of just the smaller subset of analytes as requested. (E.g., KWF received extra data for free. In this case., there are no consequences of deviating from the planned analyses).


4.) Do the laboratory reports provide results for all sites and parameters?

The laboratory reports provide results for all sites, and for all parameters, with the exceptions outlined above in question #3.


5.) Is a copy of the Chain of Custody included with the laboratory reports?

We worked with three separate laboratories in 2021:

  • SGS Laboratories, Anchorage, AK

    • Chain of Custody documents are included within the PDF laboratory reports linked above earlier in this appendix.
  • Soldotna Wastewater Treatment Plant, Soldotna, AK

    • Chain of Custody documents are on file with Kenai Watershed Forum for fecal coliform and total suspended solids for 5/11/2021, and for total suspended solids on 7/27/2021.
  • Tauriainen Engineering & Testing, Soldotna, AK

    • An individual document for each sample reports the time and date of delivery and analysis for each sample. These documents are included with the PDF laboratory reports linked above earlier in this appendix.


6.) Do the laboratory reports match the Chain of Custody and requested methods throughout?

The laboratory reports match the Chain of Custody and requested methods, with the one exception discussed in question #3. For summer 2021, the SGS laboratory performed the 200.8 analyses for all 27 analytes available for the method; instead of just the smaller subset of analytes as requested. (E.g., KWF received extra data for free. In this case., there are no consequences of deviating from the planned analyses).


7.) Are the number of samples on the laboratory reports the same as on the Chain of Custody?

The quantity of sample bottles sent to the laboratories matches the number of analyzed samples for samples collected and delivered on 5/11/2021 and 7/27/2021.


8.) Was all supporting info provided in the laboratory report, such as reporting limits for all analyses and definitions?

We worked with three separate laboratories in 2021:

  • SGS Laboratories, Anchorage, AK

    • SGS provided data as PDFs which included reporting limits, as well as in the form of an Electronic Data Deliverable where this information is also included in column format.
  • Soldotna Wastewater Treatment Plant (SWWTP), Soldotna, AK

    • SWWTP provided data in the form of .xls files. Reporting limits are described in this project’s current Quality Assurance Action Plan.
  • Tauriainen Engineering & Testing, Soldotna, AK

    • Tauriainen provided data in the form of PDF documents. Reporting limits are described in this project’s current Quality Assurance Action Plan.


9.) Are site names, dates, and times correct and as expected?

  • Yes, after post-season correction documented in this report. Notes: In 2021 Kenai Watershed Forum used pre-printed waterproof labels on all sample bottles, reducing opportunity for field and lab transcription errors. Remaining site name transcription errors from laboratories were corrected in post-season data review.


10.) Were there any issues with instrument calibration?

  • Instruments to measure intrinsic water quality parameters (sondes; to measure pH, dissolved oxygen, conductivity, turbidity) were not employed in 2021.

  • Teams did use hand-held probes to record water temperature on-site. Prior to field use, the hand-held probes were verified as measuring within the accuracy level defined in the QAPP using an ice bath in the Kenai Watershed Forum laboratory.


11.) Did the instruments perform as expected?

  • The hand-held water temperature probes performed as expected in 2021.


12.) Was instrument calibration performed according to the QAPP and instrument recommendations?

  • Water temperature is a parameter that is “verified” rather than calibrated. The hand-held water temperature probes were verified as measuring within the accuracy level defined in the QAPP using an ice bath in the laboratory, according to instrument recommendations.


13.) Was instrument verification during the field season performed according to the QAPP and instrument recommendations?

  • The hand-held water temperature probes were verified as measuring within the accuracy level define in the QAPP using an ice bath in the laboratory, according to instrument recommendations.


14.) Were instrument calibration verification logs or records kept?

  • Yes. These records are held at Kenai Watershed Forum, 44129 Sterling Hwy, Soldotna, AK.


15.) Do the instrument data files site IDs, time stamps and file names match?

  • Instrument files were not employed in 2021. Measurements from hand held probes were recorded on waterproof paper field forms.


16.) Is any insitu field data rejected and why?

  • No insitu data is rejected from 5/11/2021 or 7/27/2021.


17.) Were preservation, hold time and temperature requirements met?

  • Yes. Summer and Spring 2021 holding time requirements were met for all samples. See downloadable files below. Laboratory result documents indicated no compromises of preservation and temperature requirements.
Joining with `by = join_by(epa_analysis_id)`


18.) Are dissolved metal quantities less than total metals quantities?

  • Dissolved metals results are not consistently less than total metals results, as would be logically anticipated (see post from Flowlink Environmental website for description and rationale):

    • “Dissolved metal concentration is determined by filtering a water sample through 0.45 uM filter. Water that passes through the filter is analyzed for metals and the result is reported as dissolved metal concentration. Total metal concentration is determined by analyzing the unfiltered sample.

      “Total metal concentration = Dissolved metal concentration + Particulate (insoluble) metal concentration.

    • In 2021, there are results available for both dissolved and total metals for three elements: Ca, Fe, and Mg; for sites at and downstream of RM 31.

    • Dissolved metals results are available for 7/27/2021 (summer) only, and not for 5/11/2021 (spring); see question #3 in this sequence.

  • Results:

    • Calcium: all available observations show dissolved metals > total metals

    • Iron: all available observations show dissolved metals < total metals

    • Magnesium: nearly all available observations show dissolved metals < total metals, with the exception of three sites, all near the Kenai River mouth


As of Fall 2023 it is unclear why the values measured for total metals is greater than dissolved metals for the cases described above.

See discussion XX in “Summary” section of this report for plans in future years to address the issue of dissolved metals values being greater than total metals values in some cases.


19.) Are the duplicate sample(s) RPD (Relative Percent Difference) within range described in QAPP?

In 2021, field duplicate samples were collected at two sites on both 5/11/2021 and 7/27/2021:

  • No Name Creek (Tributary, RM0)
  • Funny River (Tributary, RM30)

Following the example in the Alaska Department of Environmental Conservation 2021 Kenai River Metals Field Report (Apsens and Petitt 2022): “… a set of paired samples was evaluated for RPD only if:

  • a.) one or both of the samples were above the Limit of Quantitation (LOQ); AND if
  • b.) one or both of the samples were at least two times the LOQ.”

A csv file of Relative Percent Difference values organized by site, date, and parameter may be downloaded below:


From the above downloadable table, we can see that a limited subset of paired values from field duplicates are eligible to be used for RPD calculations for 2021. Pairs of measurements must meet LOQ criteria described above (“a” and “b”) in order to be used. For 2021 data, of 69 paired field duplicate observations, 24 of them meet these criteria and thus may be used for RPD calculations.

RPD values range from 0 % to 165.22 %.

In the 2020 Quality Assurance Action Plan, Relative percent Difference values for most parameters are set at 20%, with the exception of “Phosphorus, total” and “Nitrates (NO2+NO3)” at 25%, and “Suspended solids, total” at 5%.

Using the available RPD values against the above criteria, four paired observations exceed project precision goals in 2021. Three of these four observations are for fecal coliform, and one is for total phosphorus.

Most other parameters have at least one available RPD value that is within project precision goals defined in the QAPP. The parameters that do not have at least one RPD calculation available, due to the LOQ constraints described above, are the following:

  • Arsenic
  • Cadmium
  • Chromium
  • Copper
  • Lead
  • Zinc

Other notes

In 2021 the full suite of dissolved metals was run for method 200.8 on samples from 7/27/2021. However we intend to only submit data for parameters that had a pre-defined QA plan as outlined in the QAPP, which is the case for the following dissolved six metals: Mg, As, Cd, Cr, Cu, Pb, and Zn.

Summary of Discussion on RPD (Relative percent Difference) Values for 2021

We attempted to calculate RPD values for all parameters collected in 2021 at two field sites on two dates. For each pair of duplicate observations, one of the following descriptions apply:

  • It was possible to calculate RPD values because:
    • a.) one or both of the samples were above the LOQ; AND one or both of the samples were at least two times the LOQ
    • Project data for this parameter will be submitted to EPA WQX if RPD values meet project goals outlined in the QAPP
  • It was not possible to calculate RPD values because:
    • a.) one or both of the samples were not above the LOQ; OR one or both of the samples were not at least two times the LOQ
    • Project data for this parameter will be submitted to EPA WQX if both paired values are non-detect

Best practices for interpreting and applying RPD data in order to flag/not flag the overall dataset are currently being finalized. KWF communicated with the Alaska Department of Environmental Conservation on August 22, 2023 via an inquiry on this topic. The memo includes a visual flow chart of the logic for when data is flagged, and is available for download below.


Final determinations on flagging / not flagging 2021 data submitted to the EPA WQX

All 2021 field results will be submitted unflagged to the EPA WQX unless discussed below.

  • Fecal Coliform

    • Given that 3 out of 4 RPD values for fecal coliform values are >60%, field data values for the 2021 dataset may vary from actual environmental conditions. The 2020 KWF QAPP does not specify an RPD value for fecal coliform, it only specifies control checks for sterility and temperature. We used EPA Method 9222D (drinking water) for fecal coliform analysis methods, which does not specify an RPD value. However EPA general recommendations for pathogens in sewage sludge recommends and RPD of 20% (link).

    • Given the above considerations, KWF will submit 2021 fecal coliform data to the EPA WQX as “flagged”. Fecal coliform data will be evaluated in this report and assessed against state standards, but given the high RPD values among duplicate samples, 2021 data will likely not be considered in the context of potential regulatory exceedences.

    • TO DO: Calculate and visualize RPD values for 2014-2022 and compare, also compare to RPD values in previous comprehensive report.

  • Selected Metals (As, Cd, Cr, Cu, Pb, Zn)

    • Because these parameters do not have an associated RPD value from 2021 field data, they will be submitted to EPA WQX as “flagged.”
    • Note: beginning in Summer 2022, KWF initiated a series of changes to the QAPP that will better facilitate QA/QC decisions regarding Total Metals and Dissolved metals results. These changes include inclusion of a dissolved metals field blank, and changing from field filtration to lab filtration as standard practice.
Joining with `by = join_by(activity_start_date, characteristic_name)`


20.) Were there any laboratory discrepancies, errors, data qualifiers, or QC failures (review laboratory duplicates, matrix spikes and blanks)?

Laboratory discrepancies from the SGS lab are described on page 2 of the spring and summer PDF document reports, which are available for download below.


The following discrepancies were identified in the results from SGS laboratories:

Matrix Spike Recoveries

A total of 6 matrix spike or matrix spike duplicate samples are outside of QC criteria. The limit of recovery range for the analyte “Total Nitrate/Nitrite-N” is 90% - 110%. For these matrix spike measurements outside of the QC criteria, recovery levels range from 112% - 118%. These values that exceed threshold may be downloaded as a table below:


No additional laboratory QA/QC anomalies for any other parameters were noted by any laboratories contracted in 2021. Additional details are available upon request at

As of 2023-11-02, “Total Nitrate/Nitrite-N” values will be flagged for the spring 2021 dataset. (Spring only and not summer because no matrix spike recovery exedences were observed from the summer 2021 lab results). The matrix spike recovery exceedances indicate that reported lab values may be 2% - 7% above actual field values. At a later date we will evaluate these results in the context of year 2000 - present data to help evaluate the relevancy of this flag.


21.) Is any laboratory data rejected and why?

No additional 2021 laboratory data currently indicates a need for being flagged or rejected.


22.) During the field season, review raw data files (EDDs [Electronic Data Deliverables], instrument records) as they are received. Document changes and corrections to methods as needed.

In 2021, KWF initiated as standard practice the receipt of machine-readable EDD files from contracted laboratories, in addition to PDF results. The EDD files are being reviewed as part of this QA/QC process, which was initiated in Winter 2022.


23.) Is the dataset complete and did you receive the expected number of results?

From the discussion and results in question #3, deviations are evident between planned and actual results reported. The reasons for the deviations are known and are attributable to the two factors discussed in question #3, and thus not concerning to project integrity.


24.) Was the data collected representative of environmental conditions?.

The data collected in this project is representative of seasonal hydrological conditions on two fieldwork days in 2021. Conditions vary instantaneously thus this dataset serves to provide representative longitudinal snapshots of the watershed across space on the two dates. Flagged data may indicate that results may require additional interpretation or exclusion from analysis.


25.) Does project meet Completeness Measure A criteria?

From the QAPP, Completeness Measure A: “the primary number of samples collected divided by the usable number of samples submitted to ADEC with a goal of 85% completeness.” (Note that as of 2023 results are now submitted directly to EPA rather than DEC).

(Note: metals parameters not specified in the QAPP are excluded from these analyses. In 2021, SGS laboratories provided dissolved metals results for parameters beyond what was originally requested.)

We calculated Completeness Measure A at the following project scales:

  • For individual parameters
  • For individual sites
  • As a project whole



Discussion on “Completeness Measure A” Results

Based on the data flagging decisions discussed in the previous questions, the 2021 dataset did not achieve all Completeness Measure A goals.

When completeness results were summarized by parameter, all dissolved metals had a value of 0% for Completeness Measure A since the decision was made to flag these as a result of an inability to calculate Relative Percent Difference values.

When completeness results were summarised by Site, Completeness Measure A values ranged from 44.4% - 71.4%, depending on the site’s sampling plan.

When completeness results are summarized for the overall 2021 dataset, Completeness Measure A is calculated at 50.1%; indicating that 50.1% of the overall result values were flagged.


26.) Does project meet Completeness Measure B criteria?

From the QAPP, Completeness Measure B: “the planned number of samples divided by the useable samples (submitted to ADEC) with a goal of 60%.”

For these calculations, we will consider “usable” as “unflagged

  • For individual parameters (analysis methods)
  • For individual sites
  • As a project whole

Work still in progress on calculating Completness Measures B as of 2023-11-02


27.) Was the QA officer consulted for any data concerns?

Note: the 2012 and 2019 approved Quality Assurance Action Plan (QAPP) documents list Alaska Dept of Environmental Conservation Staff as the project’s QA officer (Kenai Watershed Forum 2012, 2019), whereas the 2023 approved QAPP lists Kenai Watershed Forum staff as the project QA officer(Kenai Watershed Forum 2023).

Kenai Watershed Forum consulted regularly with staff from the Alaska Department of Environmental Conservation in order to prepare 2021 field data and confirm methods.


28.) Are the correct monitoring locations associated with the project? Are latitide and longitude filled out in a consistent format?

In the above map we can see that coordinates for the 2021 field sites correspond to the locations listed in the Quality Assurance Project Plan, thus they are filled out in a consistent format.


29.) Are the QAPP and other supporting documents attached?

The QAPP will be included and attached in the data submission to the EPA WQX. Find the current approved QAPP posted on Kenai Watershed Forum’s website at https://www.kenaiwatershed.org/news-media/qapp-revisions-completed-2023/.


30.) Is all project metdata correct?

Answer TBD.


31.) Is the organization ID correct?

Kenai Watershed Forum’s Organization ID in the EPA Water Quality Exchange is “EPA_WQX.” This value is manually confirmed during the upload process.


32.) Are the time zones consistent and correct (AKDT in summer)?

All times zones are listed as AKDT.


33.) Are all media types included? Media types appropriate to Characteristic?

“Water” is the only media type considered in this project. All “Characteristics” (parameters) are measured from water samples or in-situ water conditions.


34.) Check Sample Collection, Preparation and Preservation Methods, Thermal Preservative, Equipment ID, Activity Media. Is supporting information included and correct??

Work in progress here as of 2023-11-02


  1. https://dec.alaska.gov/water/water-quality/integrated-report/↩︎